Quantum critical points of Helical Fermi Liquids 
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Following our previous work, we study the quantum phase transitions which spontaneously de- 
velop ferromagnetic spin order in helical fermi liquids which breaks continuous spin-space rotation 
symmetry, with application to the edge states of 3d topological band insulators. With finite fermi 
surface, the critical point has both 2 = 3 over-damped and z = 2 propagating quantum critical 
modes, and the z = 3 mode will lead to non-fermi liquid behavior on the entire fermi surface. In 
the ordered phase, the Goldstone mode is over-damped unless it propagates along special direc- 
tions, and quasiparticle is ill defined on most parts of the fermi surface except for special points. 
Generalizations of our results to other systems with spin-orbit couplings are also discussed. 

PACS numbers: 



Helical fermi liquids (FL) have momentum dependent 
spin or pseudospin alignments at the fermi surface. For 
instance, the well-studied Rashba model P, |3] 



H 



2m 



(1) 



have inner and outer fermi surfaces with opposite inplane 
helical spin direction. Another example of helical FL is 
the edge states of 3d topological band insulators (TBI) 
like Bi2_a;Sn3,Te3 [3-8], which can be described by the 
following Dirac fermion Hamiltonian 



(2) 



Vf is the fermi velocity at the Dirac point. When the 
chemical potential is nonzero, the spin ct" of the elec- 
trons are perpendicular with their momenta at the fermi 
surface (Fig. [1]). Eq. [2] is the minimal model of helical 
FL because it has only one fermi surface, and the time- 
reversal partner of this model is located on the opposite 
edge of the three dimensional TBI. Recent ARPES mea- 
surement has successfully observed the helical spin 
alignment of the edge states of 3d TBI. Both the Rashba 
model and Eq. [5] are invariant under the following sym- 
metry transformations: 



t, ki 



X — > —X, (7 a , a" ^ —a" 



y — > —y, a" tr" 
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T, Pa are discrete symmetry transformation, while Rg 
continuously rotate spin and space by the same angle 9, 
which corresponds to the conservation of total angular 
momentum. In this work we will take the edge states of 
the TBI as an example of helical FL, but our results can 
be straightforwardly generalized to other situations. 

Just like the ordinary fermi liquid, strong enough inter- 
action can lead to various types of instabilities of helical 



FL, which spontaneously break all or part of the sym- 
metries listed in Eq. [3l Supposedly strong interaction 
will play an important role in the TBI with transition 
metal elements, where the interplay between spin-orbit 
coupling and interaction can lead to many interesting 
phenomena fol. fiol. According to the standard Hertz- 
Mihis theory [nnil, for ordinary fermi liquid, the quan- 
tum critical modes are usually over-damped due to low 
energy particle-hole excitations, which lead to nonrela- 
tivistic universality class. In a recent paper we studied 
the discrete time reversal symmetry breaking of the he- 
lical FL and the helical spin alignment at the fermi 
surface strongly suppresses the coupling between order 
parameter and the particle-hole excitations. Therefore 
the T-breaking phase transition belongs to the z = 1 
3d Ising universality class. In the current paper we will 
study the phase transition that breaks the continuous 
symmetry Rq in Eq. [3l which is associated with the in- 
plane ferromagnetic spin order (j) = {(j)x,4>y)- Identifying 
the leading spin order instability of the helical FL re- 
quires the detailed knowledge of the fermion interaction. 
As in our previous work, we will focus on the univer- 
sal physics at the quantum critical point, assuming the 
existence of the phase transition. 

Without loss of generality, the Lagrangian describing 
this transition can be written as 

L = Lf + Lb + Lbf, 
Lf — 'ip\{idt - fJ.) + iv f z ■ (a X V))ip , 



(4) 



The order parameter couples with the Dirac current, 
which is similar to the 3d QED with gauge field — 
{(f)y, —(jix, 0) and the temporal gauge choice oq = 0. The 
temporal component ap represents the charge density 
mode, which will couple to the XY spin mode after inte- 
grating out fermions. The implication of the spin-charge 
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FIG. 1: The fermi surface of helical FL, with finite chemical 
potential. The green dashed arrow is the direction of mo- 
mentum q carried by the blue arrow is the direction of 
0, the red arrow represents the helical spin direction on the 
fermi surface. (Left), a longitudinal mode of with </> parallel 
to q interacts strongly with the helical fermi liquid close to 
Kj _L q, where the spin is parallel to 0; (Right), a transverse 
mode of </> interacts weakly with helical fermi liquid. 



coupling has been studied in Ref. [lj,ll5|- However, at 
the frequency and momentum range we are interested in, 
the charge mode will not lead to singular corrections to 
the spin response function, i.e. the charge mode is not 
critical. The transition occurs when r = 0. Let us take 
/i = first. Lfc alone describes a 3d XY transition, but g 
is obviously relevant at the 3d XY and free Dirac fermion 
fixed point based on the well-known scaling dimensions 
[■0] = 1 and [(^] = 0.519 [13], this fermion-boson coupling 
will modify the nature of this transition. A controlled 
starting point for the calculation of critical exponents, is 
to increase the number of fermion components to iV > 1, 
and take the large-iV limit. After integrating out the 
fermions, the renormalized boson Lagrangian reads 



Pah 



9 i ? 9 ' 
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In the large- limit the scaling dimension of 4> is [(j>] = 1. 
A standard expansion calculation can be applied to 
the case with large but finite A^, although when A^ 1 
there is no small parameter for expansion. Phase tran- 
sitions with order parameters coupled to certain com- 
ponent of Dirac current were studied in the context 
of c?— wave superconductor by e expansion 17, [l3| and 
1/A^ expansion [3, [i^l, and a fixed point with extreme 
anisotropic fermi velocity was found if we start with an 
anisotropic initial condition l^, 2o| . 

Now let us consider the situation with /i 7^ 0, i.e. 
the situation with finite fermi surface. In the ordinary 
fermi liquid, an order parameter with small momentum 
\q\ kf interacts most strongly with fermions aX Kf _L g , 
because there the particle-hole excitation at momentum q 
is softest, and usually leads to over-damping of the quan- 
tum critical modes. In our current case, since the spin 



alignment at the fermi surface is determined by its mo- 
mentum, not all quantum critical modes have strong in- 
teractions with the fermions. For instance, for a quantum 
critical mode with and momentum g both parallel to i, 
it couples with the fermions at two points Kf = (0, ikf) 
in the same way as the ordinary fermi liquid, therefore the 
longitudinal mode of (f> is over-damped (Fig. [T]). If is 
parallel with y while q parallel with i, since the matrix 
element (i/'felf^lV'fe) = when k — (0, ±fc/), the trans- 
verse mode of 4> should couple weakly with the fermions. 
These observations suggest that after integrating out the 
fermions, the transverse and longitudinal modes of (j) will 
behave differently. Indeed, the Lf + Ltf part of the La- 
grangian Eq. |4] is invariant under gauge transformation 



0a ^> 0a + eabdbO, V ^ e'V, 



(6) 



and 9 is an arbitrary function of space. If we integrate 
out the fermions, and consider a Feynman diagram with- 
out boson internal line, this gauge symmetry implies that 
any external boson line of this diagram only involves the 
longitudinal mode of when the frequency of this exter- 
nal line is zero. For instance, Eq. [5] is consistent with 
this conclusion, because Pab^bcqc — when w = 0. 

The above observation becomes explicit in the bubble 
diagram in Fig. [2jz, which renormalizes the Gaussian 
part of Lb as 



AL,, 



(7) 



If we fix chemical potential /i and the energy at the ul- 
traviolet cut-off, the calculation suggests that the static 
and uniform susceptibility x(Oi 0) vanishes, which can be 
naturally expected because a uniform order of merely 
moves the entire Dirac cone away from the original po- 
sition, without developing any extra polarization on the 
fermi surface. This calculation implies that the mass gaps 
of transverse and longitudinal modes are still equal after 
coupling to the fermions, because a different mass gaps 
for these two modes will lead to very singular long range 
interaction between in real space-time. In the ordinary 



Hert-Millis theory [ll|, |12| of quantum phase transition 
inside fermi liquid, the most singular correction to the 
effective Lagrangian of order parameter comes from the 
imaginary part of the susceptibility, which corresponds 
to the damping of the critical mode of order parameter 
through particle-hole excitations. The damping rate 
can be calculated from the Feynman diagram Fig. [21', or 
through the Fermi-Golden rule 

/(Pk 
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propagator of (/) reads 



FIG. 2: a, the bubble diagram for the self-energy correction 
to (/>; b, the self-energy correction to fermions. 
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Matrix Pat projects to its longitudinal part, and P 
satisfies the algebraic relation P^ = P. This calculation 
indicates that only the longitudinal part of (p is over- 
damped, while the transverse part of (p gains a much 
weaker damping as lo/ [vfq) 0, which is consistent with 
our observation. The decomposition between transverse 
and longitudinal modes also occurs in a very different 
problem: the nematic transition of fermi liquid 21 1. 

The real part of self-energy can be calculated accord- 
ingly. The result in the infrared limit depends on how we 
take the limit of the frequency and momentum. In the 
limit uj/{vfq) <C 1, the result reads 



£k [f{ek+q) - f{ek)]{ek+q - £fc) 

F{9)ab 



dO- 



v'.q-' 



-g^^Pab + 9^^{6ab ~ Pab) 
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F{9)ab is a spin dependent function of 9 (the angle be- 
tween k and q). The ellipses include less singular terms. 
The transverse mode and longitudinal mode both ac- 
quire singular correction io'^/q^. The uP' jq^ behavior is 
the reason of the existence of various collective modes 
of the ordinary fermi liquid, such as the zero sound. In 
our previous paper about the Ising transition [l^, the 
up' /q^ term does not show up because the matrix element 
(fc|(T^|A; + 9)1^ ^ q^ with small g, which cancels the q^ 
in the denominator. Keeping all the relevant terms from 
imaginary and real parts, the full Gaussian Lagrangian 
for in the Euclidean space-time reads 

Lb ~ [Ag^^^ +vfq^)Pab4'a.-L0. -{1^,1^4 
vjq 

2 

+ {Bg^^ + V^q^){6ab - Pab)(j)a,-u.,-q(l)b,u^,i^O) 

Vjq 

A and B are order one dimensionless constants, vi and 
Vt are renormalized boson velocities. The renormalized 
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This calculation suggests that by coupling to the helical 
FL, at the quantum critical point the longitudinal part of 
(j) becomes a. z — i over-damped mode, while the trans- 
verse part of (j) becomes a z = 2 propagating mode. For 
both z = i and z = 2 scaling, uj/q Q with small fre- 
quency and momentum, which justifies the small w/ (u/q) 
limit we took at the beginning of our calculation. Because 
x(0, 0) vanishes, both transverse and longitudinal modes 
become gapless at the same critical point. 

In two dimension, z — 2 quantum critical point is at 
the upper critical dimension, therefore the Gaussian fixed 
point Eq. [TU] is stable with marginally irrelevant per- 
turbations. As discussed in ordinary z — Z and z = 2 
quantum critical points, more singular perturbations may 
be gene rated with higher order fermion loop diagrams 
22I |23| . In principle the stability of the one-loop re- 
sult requires either careful analysis of the higher loop ex- 
pansions or nonperturbative approach as Ref. [24| . We 



will discuss this in future, right now we tentatively focus 
on the one- loop result. The z — Z and z = 2 quan- 
tum critical modes define the quantum critical regime 
T > |r|^'^ ^ |7.|3/2 ^ j-ggpec^iygiy^ therefore 

in the quantum critical regime at low temperature the 
thermal dynamics is dominated by the z — S longitudinal 
mode, which leads to the specific heat scaling Cv T^/s 
In the quantum critical regime, the fermions interact 
strongly with the bosons, and gain self-energy renormal- 
ization. The self-energy renormalization can be calcu- 
lated in the standard way using diagram Fig. Let 
us take the quasiparticle at Kf — {+kf,0), where the 
dispersion of quasiparticle can be expanded as ej, ~ 
Vfkx + Vyky, therefore around this point k^ has scaling 
dimension 2 while ky has scaling dimension 1. Since the 
spin is along the y direction, in the calculation we should 
project the boson propagator in the y direction. Using 
the propagator in Eq. 111! the fermion self-energy reads 



cPkdv 



1 



(27r)3 i{uj + v) - Vf{kx + qx) - vl{ky + qyY 



1 - P, 



Ag^^,+v^k^ Bg'-: 



fc2 



i;2fc2 ' 



(12) 



The self-energy is dominated by the z = Z longitudinal 
mode, which leads to the same self-energy scaling as the 
ordinary z — i quantum critical point: 



E^(c.)"^ |u;p/3sgnM 



(13) 



Therefore at the quantum critical point the system has 
non- fermi liquid behavior. 

It is useful to discuss more about the isolated fermi 
patch around Kf = (-|-A:/,0). As in Eq. [T^l after pro- 



4 



jecting the boson propagator along y direction, the trans- 
verse part of the propagator acquires a factor 1 — Pyy — 
^xli^x + ^y)- Because now has scahng dimension 2, 
the transverse part of the propagator is effectively sup- 
pressed. Therefore in the Lagrangian we can keep just 
the scalar longitudinal mode. Now the isolated patch is 
described by the following scaling invariant Lagrangian 



L = Lf + Lb 



Lf = ^(c|w|2/3sgnM -t-//c, -Wj,fc2)^t ^^^^ 



Lb 
Lbf 



uj.k 



(14) 



One can verify that under the scaling transformation ui — >■ 
Lub^ , kx — !> kxh"^ ^ ky kyb, ip ipb"^, 4> ^ 'pb'^, 9 ~^ 9 
this Lagrangian is invariant. Therefore the coupling g is a 
marginal perturbation, and the theory becomes identical 
to the spinon and gauge field problem discussed in Ref. 
isl 26| . In the large- limit with N copies of the fermi 
patches, this theory is expected to be controlled by a 
strongly coupled CFT [251. 

Now let us discuss the ordered phase, with r < in 
Eq. m The low energy physics of the ordered phase is 
dominated by the Goldstone mode. Let us assume the 
order is (cj)) ~ (0,(^y), and the Goldstone mode is (j)x- 
As already mentioned before, in the ordered phase, the 
entire Dirac cone is translated in the momentum space 
without change of the shape of fermi surface, therefore 
the Goldstone mode of ordered phase of (p has a very 
similar behavior as the quantum critical mode: 



Ag 



Bg' 



r 



When the momentum is along x (longitudinal), this 
Goldstone mode is an over-damped z = 3 mode; when 
momentum is along y (transverse), the Goldstone mode 
is a propagating z = 2 mode. In the ordered phase, the 
over-damped longitudinal Goldstone mode will lead to 
the same nonfermi liquid self-energy correction as Eq. 
[13] for almost all points on the fermi surface: 



^i,{uj)" 



|cos(0)|*/3|^|2/3ggj^[^] 



(16) 



(f) is the angle between Kf and 0. At the special points 
(f) — ±7r/2, the correction to the fermion self-energy 
mainly comes from the transverse part of the Goldstone 
mode, which reads 



S^(w)" 



|wp/^sgn[a;] 



(17) 



Therefore in the ordered phase only two special points of 
the fermi surface have well-defined quasiparticles. The 



critical dynamics and the fermion self energy behavior in 
the ordered XY phase are similar to the nematic transi- 
tion in 2d fermi liquid 21 1 . 

So far we only kept the lowest order momentum depen- 
dent terms in Eq. [2J while in real system the symmetry 
Re is broken by lattice symmetry, as was shown by first 
principle calculations [27[. In Bi2_a;Sna;Te3 the continu- 
ous 0(2) symmetry of Rq is broken down to Ce inplane 
rotation symmetry with large chemical potential Q , and 
the spin will be canted along z direction except for iso- 
lated points on the fermi surface 27|, |28| . In our previ- 



ous work we have argued that the z direction canting will 
lead to damping of the Ising order parameter (p ~ tp'^a^tfj. 
The XY order parameter will still be decomposed into 
damped part and undamped part, although both parts 
will only have discrete rotation symmetry. 

We have used the edge states of 3d TBI as the example 
of helical FL, our analysis is applicable to other helical 
FL. For instance, if we give graphene a small but finite 
chemical potential, we can consider the spontaneous gen- 
eration of order ^pjT°'ip. T" S SU(4) is the flavor sym- 
metry matrix operating on the real spin and Dirac cone 
valley space. The gamma matrices which operate on 
the two sublattices plays the role as the helical spin in our 
analysis, and the results in our paper are still applicable. 
Our study can also be applied to the Rashba model in Eq. 
[U as long as the momentum q carried by the boson field 
(pq' is much smaller than the distance between the two 
fermi surfaces of Rashba model: q <^ |i?/,o«t| — |^/,m|, 
therefore (j) does not induce coupling between the two 
fermi surfaces. One important difference between the 
Rashba model and the edge states of TBI is that, the 
uniform and static susceptibility is nonzero for Rashba 
model, although the transverse and longitudinal modes 
still gain the same uniform and static correction. 

Generalization of these results to other spin-orbit cou- 
pled electron models is straightforward. The most gen- 
eral form of spin-orbit coupled electron model is 



(18) 



Here the two bands of the Pauli matrices can stand for 
different physical degrees of freedom depending on the 
context, for instance this Hamiltonian can also repre- 
sent hybridization between different orbital states. When 
B{k)a takes the p-wave form Ba ~ Cabkb, the model be- 
comes the Rashba model. For example, let us consider 
the following model as a 3d version of Rashba model: 
Ba ~ ka, a = x,y,z. This model also has inner and outer 
fermi surfaces with hedgehog spin distribution around 
the fermi surface. The system has 0(3) rotation sym- 
metry as long as spin and space rotation are synchro- 
nized. It would be interesting to consider the sponta- 
neous breaking of this continuous symmetry by develop- 
ing nonzero order oi cp = ((px,<py,(pz)- After integrating 
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out the fermions, just like the two dimensional cases we 
considered above, the vector <j) is decomposed into one 
longitudinal mode and two transverse modes. At the 
quantum critical point, the longitudinal mode is a z = 2 
propagating mode while the two transverse modes are 
z — i over-damped modes. We can also consider the 
possibility of d-wave B{k)a, for instance ^ — ky, 
By 2kxky, which can be realized in hole-doped GaAs 
quantum well with inversion symmetric confining poten- 
tial [29] . The result of our paper is still applicable to this 
model as long as we define the projection matrix in Eq. 
[10] and Eq. [nlas Pab - BaBb/{Bl + Bl). 

As a summary, in this work we discussed the spon- 
taneous breaking of the continuous spin-space combined 
rotation symmetry in helical fermi liquid, we calculated 
the quantum critical modes at the quantum critical point, 
and the Goldstone mode in the ordered phase, as well 
sa their effects on the quasiparticles. However, as was 
mentioned already, higher order loop diagrams may lead 
to more singular momentum and frequency dependent 
terms which have the potential to destroy the Gaussian 
fixed point studied in our paper. Therefore the analysis 
of the Gaussian fixed point in this paper is the basis of 
our future studies. 
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